Device and method for spatially resolved photodetection and demodulation of modulated electromagnetic waves

ABSTRACT

A device and method for spatially resolved photodetection and demodulation of temporally modulated electromagnetic waves makes it possible to measure phase, amplitude and offset of a temporally modulated, spatially coded radiation field. A micro-optical element ( 41 ) spatially averages a portion ( 30 ) of the scene and equally distributes the averaged intensity on two photo sites ( 51.1.51.2 ) close to each other. Adjacent to each of these photo sites ( 51.1 ) are two storage areas ( 54.1, 54.2 ) into which charge from the photo site can be moved quickly (with a speed of several MHz to several tens or even hundreds of MHz) and accumulated essentially free of noise. This is possible by employing the charge-coupled device (CCD) principle. The device combines a high optical fill factor, insensitivity to offset errors, high sensitivity even with little light, simultaneous data acquisition, small pixel size, and maximum efficiency in use of available signal photons for sinusoidal as well as pulsed radiation signals. The device and method may be used in a time-of-flight (TOF) range imaging system without moving parts, offering 2D or 3D range data.

FIELD OF THE INVENTION

The invention relates to a 1-dimensional (1D) or 2-dimensional (2D)device and a method for spatially resolved photodetection anddemodulation of temporally modulated electromagnetic waves. It makespossible to measure phase, amplitude and offset of a temporallymodulated, spatially coded radiation field. Preferential use of theinvention is in a time-of-flight (TOF) range imaging system withoutmoving parts, offering 2D or 3D range data. Such a range camera can beused in machine vision, surveillance, all kinds of safety applications,automated navigation and multimedia applications. The invention isespecially useful in distance-measurement applications where highdistance accuracy is necessary also for objects far away from themeasurement system, in particular applications that need a distanceaccuracy independent from the target distance.

In this document, the term “light” stands for any electromagneticradiation, and preferably for visible, ultra-violet or infra-redradiation.

BACKGROUND OF THE INVENTION

German patent DE-44 40 613 C1 discloses a one- or two-dimensional arrayof demodulation pixels. One pixel contains one single photo site (photodiode or CCD gate), which is connected to one or more light-protectedstorage sites (realized as CCD pixel or MOS capacitor) by electricalswitches (realized as CCD transfer gate or as transistor switch). Thephoto site integrates charge that is generated by incoming light. Afterthis short-time integration the photo charge is transferred into astorage site by activating a switch. If realized in CCD technology, thischarge addition can be performed repetitively. For a demodulationapplication the integration time is chosen to be much shorter than theperiod of the modulation signal. Thus, the device can be used to samplethe incoming modulated signal fast enough such that no temporal aliasingtakes place.

Practical realizations known so far always used the CCD principle torealize the photo site, the electrical switch and the storage sites. Toconnect one photo gate to several storage gates by more than onetransfer gate (electrical switch) always occupies space. With today'stechnologies, accessing the photo site, for example by four transfergates, forces to realize relatively large photo gates. Charge transferfrom the photo site to the storage site (response/efficiency of theswitch) is then relatively poor and slow. Additionally, practice showsthat it is very difficult to realize four transfer gates with equaltransfer efficiencies. Therefore, current realizations suffer frominhomogeneities between the single switch/storage combinations atpractical frequencies needed for time-of-flight (TOF) applications (>1MHz). These effects lead to the fact that with today's technologies theteaching of DE-44 40 613 C1 can only be used for TOF applications whenrealized or operated as one-switch-one-storage-site-device and a specialoperation mode: it has to be operated such that the sampling points areacquired temporally serially rather than in parallel. This is a seriousrestriction if TOF-measurements have to be performed of fast-changingscenes containing moving objects.

German patent application DE-197 04 496 A1 describes a similar pixelstructure consisting of at least two modulation photo gates (MPG) anddedicated accumulation gates (AG). An MPG pair is always operated in abalanced mode (as balanced modulator). The charge carriers are opticallygenerated in the depletion region underneath the MPGs by the incomingmodulated light and guided to the accumulation gates by a potentialgradient. This potential gradient depends on the control signals appliedto the MPGs.

DE-197 04 496 A1 includes a pixel realization with only one MPG pairoperated sequentially with two phases relative to the phase of themodulated transmitter and thus enabling the measurement of the receivedlight's time delay. As in practical realization of DE-44 40 613 C1, thisserial acquisition of an “in-phase” and “quadrature-phase” signalrepresents a serious drawback when being used for TOF applications withfast-changing scenes.

Additionally, DE-197 04 496 A1 suggests a realization with four MPGs andfour AGs, where always two MPGs build a balanced modulation pair, andboth pairs are operated with different phase with respect to each other.In that way, four phase-measurements (sampling points) of the incominglight can be measured in parallel. This access to the light sensitivearea from four local different places again, as is the case in DE-44 40613 C1, results in a non-uniform charge distribution and gives eachaccumulation gate a different offset, which is complicated tocompensate. DE-197 04 496 A1 suggests two different possibilities:

-   (i) The AGs are realized as CCD gates. Then the charge carriers can    be integrated under the AGs and read out by a multiplex structure,    for example a CCD, after an integration period.-   (ii) Alternatively, instead of integrating the charge the AGs can be    realized directly as pn-diodes and the signal can be read out as    voltage or current (for example with an APS structure), or these    signals directly feed a post-processing structure to measure phase    and total intensity.

Such a post-processing APS-structure, however, occupies space on thesensor and will always drastically increase the sensor's pixel size and,hence, decrease its fill factor. Additionally, feeding the generatedphotocurrent directly to an amplification stage before being integrated,adds additional noise sources to the signal and decreases thestructure's performance, especially for low-power optical input signals.

German patent application DE-198 21 974 A1 is based on DE-197 04 496 A1.Here some special dimensions and arrangements of the MPGs are suggested.The MPGs are implemented as long and small stripes with gate widths ofmagnitude of the illumination wavelength and gate lengths of 10 to 50times this magnitude. Several parallel MPG-AG-pairs form one pixelelement. All MPG-AG pairs within one pixel element are operated with thesame balanced demodulation-control signal. All AGs are properlyconnected to a pair of readout wires, which feeds a post-processingcircuit for the generation of sum and difference current. One pixelconsists of one or more pixel elements, where each pixel elementconsists of several pairs of MPGs. If one pixel is realized with severalpixel elements, the teaching of DE-198 21 974 A1 intends to operate thepixel elements in different phase relations, in particular with a phasedifference of 90° (in-phase and quadrature-phase measurement indifferent pixel-elements). Additionally, DE-198 21 974 A1 recommends theuse of microlenses or stripe-lenses to focus the light only onto the(light sensitive) MPGs. These optical structures, however, do notcorrect for local inhomogeneities in the scene detail imaged to onepixel. Such inhomogeneities, especially to be expected due to the largepixel size, lead to measurement errors. This is because the in-phasepixel elements acquire another part of the scene than thequadrature-phase pixel elements.

The main drawback of DE-198 21 974 A1 is the targeted (relatively large)pixel size between 50×50 μm² to 500×500 μm². It is therefore not suitedto be realized as a larger array of many 10,000 of pixels. The reasonfor the described long and narrow MPGs is the need for smalltransportation distances of the photo-generated charge carriers into theAGs. Only for small distances, the structure can be used fordemodulation of high modulation frequencies (increased bandwidth). Ifthe MPGs were realized with MPGs of small length and width, thephotosensitive area of each pixel would be very small with respect tothe planned space-consuming in-pixel post-processing circuitry.Realizing several long-length and short-width modulation structures andarranging and operating them in parallel and thus increasing thephotosensitive area without losing bandwidth (due to small drift waysfor the charge carriers) is an elegant way of increasing the opticalfill-factor. However, the increased fill factor can only be realizedwith larger pixels and hence the total number of pixels, which can berealized in an array, is seriously limited with the device described inthis prior-art document.

TOF distance measurement systems always use active illumination of thescene. A modulated light source is normally located near the detector.Since the optical power density on the illuminated object or targetdecreases with the square of the distance between the illuminationsource and the target, the received intensity on the detector alsodecreases with the square of the target distance. That is why themeasurement accuracy for targets far away from the sensor is worse thanthe accuracy for near targets.

Some known TOF systems are operated with square light pulses of constantamplitude during the pulse duration (cf. Schroeder W., Schulze S.,“Laserkamera: 3D-Daten, Schnell, Robust, Flexibel”, Daimler-BenzAerospace: Raumfahrt-Infrasttuktur, 1998). The receiver is realized asor combined with a fast electrical, optical or electro-optical switchmechanism, for example an MOS switch, a photomultiplier (1D) or amicrochannel plate (MCP), an image intensifier, or the“in-depth-substrate-shutter-mechanism” of special CCDs (SankaranarayananL. et al., “1 GHz CCD Transient Detector”, IEEE ch3075-9191, 1991).Spirig's, Lange's and Schwarte's lock-in or demodulation pixels can alsobe used for this kind of operation (Spirig T., “Smart CCD/CMOS BasedImage Sensors with Programmable, Real-time, Temporal . . . ”, Diss. ETHNo. 11993, Zurich, 1997; Lange R et al., “Time-of-flight range imagingwith a custom solid-state image sensor”, Proc. SPIE, Vol. 3823, pp.180–191, Munich, June 1999, Lange R. et al., “Demodulation pixels in CCDand CMOS technologies for time-of-flight ranging”, Proc. SPIE, Vol.3965A, San Jose, January 2000, Schwarte R, German patent application No.DE-197 04 496 A1.

With the transmission of the light pulse, the switch in the receiveropens. The switch closes with the end of the light pulse. The amount oflight integrated in the receiver depends on the overlap of the timewindow defined by the ON time of the switch and the delayed time windowof ON time of the received light pulse. Both ON time of the switch andpulse width are chosen to have the same length. Thus, targets with zerodistance receive the full amount of the light pulse, the complete lightpulse is integrated. Targets farther away from the light source onlyintegrate a fraction of the light pulse. The system can only measuredistances L<L_(max) within the propagation range of the light pulse,defined by half the product of pulse width T and light velocity c.

The intensity of back-scattered light decreases with the square of thetarget's distance to the emitting active illumination source. Theprior-art shutter operation leads to an additional distance-dependentattenuation of the integrated received signal:

$\begin{matrix}{{{{integrated}\mspace{14mu}{signal}} \sim {\frac{\left( {L_{\max} - L} \right)}{L^{2}} \cdot I_{trans}}},} & (I)\end{matrix}$where I_(trans) represents the transmitted light intensity;

$\begin{matrix}{L_{\max} = {\frac{T \cdot c}{2}.}} & (2)\end{matrix}$

These prior-art contents are also summarized in FIGS. 8 and 9.

In order to use this principle for performing a distance measurement,two additional measurements have to be performed: a first additionalmeasurement without any active illumination for measuring andsubtracting the background-offset, and a second additional measurementwith the active illumination switched on for measuring the amplitude ofthe back-scattered light.

SUMMARY OF THE INVENTION

It is an object of the present invention to provide a device and amethod for spatially resolved photodetection and demodulation ofmodulated electromagnetic waves that overcomes the above-mentionedlimitations of the prior art. It is a further object of the invention toprovide a device and a method for determining a distance between thedevice and a target.

The basic idea of the invention consists of using a micro-opticalelement that images the same portion of the scene on at least two photosites close to each other. Adjacent to each of these photo sites is atleast one, and preferably two, storage areas into which charge from thephoto site can be moved quickly (with a speed of several MHz to severaltens of or even hundreds of MHz) and accumulated essentially free ofnoise. This is possible by employing the charge-coupled device (CCD)principle. The device according to the invention can preferentially beoperated in two modes for two types of modulated radiation fields:

-   -   Sinusoidal radiation, signals are demodulated by operating the        two photo sites and their storage areas in quadrature, i.e., by        applying the clocking signals to the second photo site and its        storage areas with a delay of a quarter of the repetition period        compared to the first photo site and its storage areas.    -   Pulsed radiation signals are measured by switching the first        photo site's storage part from the first to the second area        during the arrival of the radiation pulse. The second photo site        and its storage areas are used for offset measurements without        emitted and received radiation.

The device for spatially resolved photodetection and demodulation oftemporally modulated electromagnetic waves according to the inventioncomprises a one-dimensional or two-dimensional arrangement of pixels. Apixel comprises at least two elements for transducing incidentelectromagnetic radiation into an electric signal, each transducerelement being associated with at least one element for storing theelectric signal, the at least one storage element being inaccessible orinsensitive to incident electromagnetic radiation. A pixel comprises anoptical element for spatially averaging the electromagnetic radiationincident on the pixel and equally distributing the averagedelectromagnetic radiation onto the transducer elements of the pixel.

The method for spatially resolved photodetection and demodulation oftemporally modulated electromagnetic waves according to the inventioncomprises the steps of

-   (a) impinging electromagnetic radiation onto a one-dimensional or    two-dimensional arrangement of pixels,-   (b) transducing electromagnetic radiation incident on a pixel into    at least two electric signals in at least two transducer elements,    and-   (c) storing each of the at least two electric signals in at least    one storage element.

Prior to step (c), the electromagnetic radiation incident on a pixel isspatially averaged and the averaged electromagnetic radiation is equallydistributed onto the transducer elements of the pixel.

The device for determining a distance between the device and an objectaccording to the invention comprises means for emitting during a firstlimited time interval pulsed electromagnetic radiation, means fordetecting incident electromagnetic radiation during a second limitedtime interval, and means for controlling the emitting and detectingmeans such that the first and second time intervals do not overlap. Thedetecting means are preferably the above-described device for spatiallyresolved photodetection and demodulation of temporally modulatedelectromagnetic waves according to the invention.

The method for determining a distance between a measurement system andan object according to the invention comprises the steps of emittingduring a first limited time interval a pulse of electromagneticradiation from the measurement system towards the object, reflectingand/or scattering at least part of the electromagnetic radiation fromthe object, and detecting during a second limited time intervalelectromagnetic radiation reflected and/or scattered from the object,whereby the first and second time intervals do not overlap. Fordetecting, the above-described method for spatially resolvedphotodetection and demodulation of temporally modulated electromagneticwaves is preferably used.

The invention combines a high optical fill factor, insensitivity tooffset errors, high sensitivity even with little light, simultaneousdata acquisition, small pixel size, and maximum efficiency in use ofavailable signal photons for sinusoidal as well as pulsed radiationsignals.

The micro-optical elements fulfill the task of averaging the receivedlight on each pixel and equally distributing the averaged light onto thetwo light-sensitive areas. This results in two optically identical photosites per pixel and makes the measurements insensitive to edges or,generally speaking, to any spatial inhomogeneity imaged to one pixel.Due to the small pixel size and the highly parallel, simplearchitecture, optical microstructures can be easily realized, produced,and assembled.

A high dynamic range can be achieved by serially taking demodulationimages with different integration times, i.e., long integration timesfor dark objects and short integration times for bright objects.Additional CCD gates for storing short-integration-time images also ineach pixel can be realized, so that the sensor does not have to becompletely read out between the acquisition of the short-time-integratedand the long-time-integrated images.

The pixel itself can be realized in charge-coupled-device (CCD)technology. Pixels can be arranged in large arrays (several 10,000pixels to several 100,000 pixels), and the readout can also be performedusing the CCD principle, for example with a “frame-transfer structure”.Alternatively, the pixel can be realized in acomplementary-metal-oxide-semiconductor (CMOS) process offering thepossibility to realize small CCD structures (3 to 20 CCD gates per pixelwith a charge transfer efficiency (CTE) of greater than 90% issufficient). With such a CMOS process each pixel can get an own readoutamplifier that can be accessed by an address decoder(active-pixel-sensor (APS) principle). Further below, more detailedrealizations in both CCD and CMOS will be introduced. The CMOS-APS/CCDrealization offers the following advantages over a pure CCD realization:

-   -   Each pixel can be addressed and read out individually. Thus        regions of interest (ROI) can be defined, for example, special        regions can get a different illumination time or readout at a        different frame rate.    -   No blooming problems like with CCDs will appear. Charge of an        overexposed pixel is dumped to a dump node or to the sense node        and does not disturb neighboring pixels.    -   No smearing problems like with CCD will appear. Instead of        moving the pixel charge through the entire imager, the charge is        converted to a voltage within each pixel.

Instead of accessing one photo site from four or more sites each pixelnow contains two photo sites. Each of these is accessed highlysymmetrically from preferably two sides by CCD gates (transfer-,modulation-, or sampling-gates). Each of these sampling gates transferscharge carriers from the photo site to a dedicated storage gateresulting in, preferentially, four isolated storage sites within eachpixel. Thus, the sampling points can be measured at the same time andthe pixels are not restricted to the observation of slowly changing orstatic processes. The device according to the invention has ademodulation bandwidth for modulated radiation fields (e.g., modulatedlight) ranging from zero to some tens or even hundreds of MHz.

Charge carriers are repetitively added and integrated in each storagesite rather than being directly fed to any post-processing electronics.This CCD charge addition ability is a nearly noise-free process andenables the system to operate with relatively low light power just byenlarging the integration times. This is an advantage over pixelrealizations with in-pixel post-processing electronics.

The pixel size can be realized smaller than possible with prior art,offering a good optical fill factor (>50% even without microlens). Thisis possible, since storing the demodulated phase information within thepixel occupies far less space than realizing additional post-processingelectronics in each pixel.

The device according to the invention can handle sinusoidally modulatedradiation signals. More than two, and preferably four temporal samplingvalues A_(i) (i=0, 1, 2, 3) are measured during subsequent, and possiblypartially overlapping, time intervals of length, for instance, equal toT/2 (where T is the modulation period), by integrating repeatedly duringthe corresponding time intervals. A₀ and A₂ are measured with the firstphoto site, A₁ and A₃ are measured with the second photo site. The phaseangle φ can be determined according to the following equation:

$\begin{matrix}{\varphi = {a\;{{\tan\left( \frac{A_{0} - A_{2}}{A_{3} - A_{1}} \right)}.}}} & (3)\end{matrix}$

The device according to the invention can also handle pulsed radiationsignals. The first photo site is switched from the first storage area tothe second storage area during the arrival time of the radiation pulse,so that a first part of the radiation signal is integrated into thefirst storage area and the rest of the radiation signal is integratedinto the second storage area. Let us call the integrated signal in thefirst storage area B₀ and the signal in the second storage area B₁. Thesecond photo site is employed for offset measurements in the followingway: the complete signal of the radiation pulse is integrated into thefirst storage area of the second photo site, producing a signal C₀. C₀equals the sum of B₀ and B₁ and therefore directly represents theintensity image. A deviation of C₀ from the sum of B₀ and B₁ can be usedto correct for optical or electrical inhomogeneities between both photosites in one pixel (calibration of the pixel). Afterwards, the secondphoto site is switched to the second storage area, and during the timein which no pulsed radiation is received, background radiation and darkcurrent is integrated into this second storage area, producing a signalC₁. The ratio

$\begin{matrix}{q = \frac{B_{0} - C_{1}}{B_{1} - C_{1}}} & (4)\end{matrix}$is a measure for the arrival time of the radiation pulse. Since neitherthe temporal form of the radiation pulse nor the temporal sensitivityfunctions of the storage areas are perfectly binary (they are rathercontinuous functions of time), the ratio q is in general a non-linearfunction of the arrival time. Therefore, it has to be calibrated withdistance measurements to produce accurate distance information.

According to the invention, the following four modes of operation withpulsed radiation signals are preferably used.

(I) Inverse Shutter Operation

The easiest way of operation according to one aspect of the invention isto start the switch at the end of the light-pulse transmission or evenlater. For the latter case a certain distance range in front of therange camera cannot be measured, since it is outside the “integrationwindow”. The open time of the switch, however, still remains the same.This leads to the fact that targets farther away from the sensorintegrate a longer period of the light pulse than objects near the rangecamera. Together with the attenuation of the back-scattered light from adiffusely reflecting target, which is proportional to the square of thedistance (assuming a Lambert-reflecting target), the resultingintegrated signal share of the light pulse now only decreases linearlywith the distance of the target. This idea is illustrated in FIG. 2.

(II) Ramp Pulse Instead of Square Pulse

Using a (falling) ramp pulse instead of a square pulse and combiningthis operation with the “inverse shutter operation” introduced in theabove section (a) results in an integrated signal which does not dependon the target's distance, only on its remission coefficient, if thetarget is a Lambert reflector. This is because the integration of thereceived linear ramp results in an integrated signal proportional to thesquare of the distance, if referenced to the totally received signalmean value. Since this received signal mean value has an inverselysquared dependence on the signal intensity on the target, the totallyintegrated signal does not depend on the distance of the target. It onlydepends on the remission characteristics of the target. Thus, thismethod can also be used to measure an object's remission (if it is knownbeing a Lambert reflector.)

Care has to be taken in the interpretation of these results. From asimplistic point of view, one might think that such a measurement wouldnot contain any information. However, one has to consider the followingfacts:

-   -   The information is only gained when referencing these results to        the “dark measurement” (no illumination) and to the “light        measurement” (non-modulated DC illumination with the active        illumination source). The result of the latter measurement        depends on the distance of the object. Illuminated objects        farther away from the camera are darker than those near the        camera. Only with these reference measurements the distance can        be calculated. The advantage is that we get a constant        signal-to-noise ratio for the modulated measurement over the        complete range. And, since the signal does not depend on the        distance for this measurement, a special high dynamic range of        the sensor is not necessary.    -   It is, of course, not true for a realistic scene that all        surfaces exactly behave like Lambert reflectors, meaning that        the light intensity decreases with the square of the object        distance. So the measurement still contains information.

Calibration of the system is still necessary. The main advantage of thisoperation mode is a decreased need for dynamic range of the sensor.Thus, this method could also find other applications where activeillumination is used to illuminate a scene and all objects should appearwith the same brightness, independent from their distance to theillumination. Amongst other applications may be cited surveillanceapplications with active illumination (e.g., IR). In this applicationthe camera wants to see both targets (e.g., a thief) far away from thecamera and targets near the camera. With conventional targetillumination, targets near the camera would lead to saturation, whichmeans no information, if the illumination is chosen such that distantobjects can be seen.

(III) Adaptation of Pulse Shape on Transfer Characteristic of Switch

Most shutter or switch mechanisms do not behave like an ideal switch.Instead, they have a typical “switch” characteristic. For instance, theydo not switch in an infinitesimally short time, but have a certain“switch” time given, e.g., by a linear or quadratic response. For somedevices this “shutter-efficiency” can be externally influenced andvaried over time, for instance, by means of an adjustable externalcontrol voltage. These changes in sensitivity have not been consideredin the descriptions done so far. A switch with linearly increasingsensitivity over time can, for example, be used in combination with apure square pulse illumination (rather than a ramped pulse), leading tothe same result as discussed in the above section (b). Also, the shapeof the light pulse can be adapted to the transfer characteristics of theswitch.

(IV) Adaptation of Transfer Characteristic of Switch

Not only the shape of the light pulse can be varied in order to changethe dependency of integrated (gated) charge from the distance, but veryoften also the detector transfer characteristic can be modified.Combinations of the operation modes (a)–(d) are also possible.

The invention overcomes the limitations of the prior art in thefollowing areas:

-   -   The three or four phase measurements (or temporal sampling        points) used for demodulation of the phase information of the        received signal can be performed simultaneously, i.e. in        parallel, rather than sequentially.    -   The overall pixel size is small. This invention is therefore        ideally suited for realization as a large array (several 10,000        pixels to several 100,000 pixels).    -   The invention offers a large fill factor of >50% (without        microlenses) and of up to 100% if microlenses are used.    -   Two optically identical photo sites are used for in-phase and        quadrature-phase signal acquisition. Therefore, the pixels are        not sensitive on local scene differences within the scene detail        imaged to one pixel.    -   Due to the simple, highly symmetrical architecture of the pixel,        a very efficient realization of a microlens structure is        possible, leading to up to an effective fill factor of 100%. In        addition, the assembly and positioning of the microlens array        becomes relatively easy because of its extension in only 1        dimension.    -   The invention is robust against device noise, since no active        electronics is used at a point of time where the signal is low.        Instead, the post-processing is performed only when the        integrated signal is strong enough.    -   The various sampling points are offset free (or at least        pair-wise offset free, depending on the realization in pure CCD        or CMOS-APS/CCD technology).    -   A TOF system can be operated with a resolution independent of        the object distance.

BRIEF DESCRIPTION OF THE DRAWINGS

The invention and, for comparison, the state of the art are described ingreater detail hereinafter relative to the attached schematic drawings.

FIG. 1 shows a basic concept of a pixel of a device according to theinvention.

FIG. 2 shows a device according to the invention.

FIG. 3 shows a pixel of the device of FIG. 2 together with an associatedmicrolens.

FIG. 4 shows an embodiment of the invention realized in CCD technology.

FIGS. 5–7 show three different embodiments of the invention realized inCMOS-APS/CCD technology.

FIGS. 8 and 9 show the principle of a pulsed-TOF distance-measurementmethod according to the state of the art.

FIGS. 10 and 11 show the principle of a first embodiment of a pulsed-TOFdistance-measurement method according to the invention.

FIGS. 12 and 13 show the principle of a second embodiment of apulsed-TOF distance-measurement method according to the invention.

DESCRIPTION OF THE PREFERRED EMBODIMENTS

The basic structure of an exemplified pixel 50 of a device according tothe invention is illustrated in FIG. 1. The pixel 50 comprises, e.g.,two light-sensitive photo sites 51.1, 51.2. A first photo site 51.1 hasa first photo gate 52.1 for the phase angle 0° and a second photo gate52.2 for the phase angle 180°; a second photo site 51.2 has a thirdphoto gate 52.3 for the phase angle 90° and a fourth photo gate 52.4 forthe phase angle 270°. Between the two photo gates 52.1, 52.2 (or 52.3,52.4, respectively) in each photo site 51.1 (or 51.2), there is a middlephoto gate 53.1 (or 53.2) which is preferably kept at a fixed potential.The pixel 50 further comprises light-protected storage gates 54.1–54.4,each of which is associated to a photo gate 52.1–52.4. A first storagegate 54.1 is for storing signals with phase angle 0°, a second storagegate 54.2 for storing signals with phase angle 180°, a third storagegate 54.3 for storing signals with phase angle 90°, and a fourth storagegate 54.4 for storing signals with phase angle 270°. The second storagegate 54.2 and the third storage gate 54.3 are isolated by a separationgate 55. All gates within one pixel 50 can be controlled separately.

In FIGS. 4–7, which show various embodiments of the invention and arediscussed further below, analogous elements are designated by the samereference signs as in FIG. 1 and will not be explained again.

The device according to the invention, as schematically shown in FIG. 2,preferably comprises a standard objective 2 for imaging a scene 1, thusgenerating an imaged scene 3. The device further comprises a microlensarray 4 and an optical application-specific integrated circuit (ASIC) 5.

The details of a pixel 40 of the microlens array 4 and a pixel 50 of theopto ASIC 5 are shown in FIG. 3. Each pixel 40 of the microlens arraycomprises an optical microstructure 41, which can be realized, e.g., asa microlens or a plurality of microlenses, as a diffractive element, ora plurality of diffractive elements, etc. The optical microstructure 41aims at equally distributing the averaged light intensity of animaged-scene pixel 30 onto the two optically identical photo sites 51.1,51.2 of the ASIC pixel 50. This is schematically illustrated in FIG. 3by choosing an example where the area 30 of the scene allocated to thepixel 50 is partially white and partially black, and where the areas31.1, 31.2 on the photo sites 51.1, 51.2 corresponding to this area 30are equally gray, represented by a cross-hatching. In the schematicrepresentation of FIG. 3, the optical microstructure 41 consists of twosub-structures 42.1, 42.2. A first sub-structure 42.1 is for averagingthe intensity, and a second substructure 42.2 is for distributing theaveraged intensity onto the two photo sites 51.1, 51.2.

The device according to the invention can preferably be realized in twodifferent technologies:

(A) the pure CCD technology; and,

(B) the CMOS-APS/CCD technology.

Embodiments realized in these two technologies are discussed below.

(A) Realization as a Pure CCD Imager

An embodiment realized in a pure CCD process is shown in FIG. 4. Onepixel 50.1 (50.2, . . . ) consists of two light sensitive areas 51.1,51.2. Each of these areas 51.1 (or 51.2, respectively) is divided intotwo or three light sensitive modulation gates 52.1, 52.2 (or 52.3,52.4); here a 3-gate realization is shown with a middle photo gate 53.1(or 53.2). During demodulation/integration operation, the middle gate53.1 (or 53.2)—if present—is kept at a fixed potential and the outermodulation gates 52.1, 52.2 (or 52.3, 52.4) are modulated in a balancedmanner. Optically generated charge carriers are then distributed to theneighboring storage gates 54.1, 54.2 (or 54.3, 54.4), depending on theactual potential gradient under the modulation gates 52.1, 52.2 (or52.3, 52.4). The storage gates 54.1, 54.2 (or 54.3, 54.4) are isolatedfrom each other by an additional transportation gate 55. The twomodulation-gate pairs 52.1, 52.2 and 52.3, 52.4 within one pixel 50.1are operated with a 90° phase difference with respect to each other, sothat the one pair 52.1, 52.2 integrates-the in-phase component and theother pair 52.3, 52.4 integrates the quadrature-phase component. Eachgate within the pixel 50.1 can be controlled individually and all pixels50.1, 50.2, . . . are operated in parallel.

The sensor is realized as a frame transfer CCD. A first, partlylight-sensitive area 56 accessible to light serves as a lock-in CCDarray for integration, and a second, light-protected area 57 serves as amemory CCD array for storage. The pixel gates are therefore operatedlike a 3-phase CCD to transfer the image into the storage CCD 57. It canthen be read out protected from further optical signal distortion.During readout, the next image can be integrated. During the imagetransfer from the first area 56 into the second area 57, smearing maytake place. This, however, does not seriously influence the measuredphase result, since all sampling points belonging to one pixel 50.1(50.2, . . . ) integrate the same parasitic offset charge. Additionally,the use of a monochromatic light source or a light source with limitedspectral bandwidth in combination with narrow band-filters canefficiently reduce background illumination and, hence, smearing effects.(The active illumination can be switched “off” during the picture shiftinto the storage CCD 57.)

Like in conventional CCDs, anti-blooming structures can be integrated inorder to prevent charge carriers of an overexposed pixel to disturbneighboring pixel information. Additionally, a charge-dump diffusion 58on top of the first area 56 enables to get rid of parasitic charge. Thedimensions of the CCD gates are preferentially chosen such that one getssquare pixels 50.1, 50.2, . . . (i.e., the gates are about 12 timeswider than long).

The proposed structure is an advantageous combination of the establishedframe-transfer-CCD principle with the new demodulation-pixel principle.

(B) Realization as a CMOS-APS/CCD Imager

The pixel can also be realized in CMOS/CCD technology with theactive-pixel concept. The CMOS-APS/CCD realization seems to be moreadvantageous than a pure CCD realization, since each pixel can beaddressed and read out individually, and blooming problems or smearingproblems do not appear. Three different embodiments realized in theCMOS-APS/CCD technology are shown in FIGS. 5–7.

FIG. 5 shows an embodiment that has only one readout stage per pixel 50.The single sampling points can then be transferred to a readout nodesequentially by operating the CCD gates like a conventional CCD line.Care has to be taken that no additional charge is optically generatedduring this transfer.

The pixel 50 comprises a dump gate 59 and a dump diffusion 60 forresetting, and an OUT gate 61 and a sense diffusion 62 for reading out.The pixel 50 additionally comprises an addressable in-pixel APS readoutcircuitry 70.

FIG. 6 and FIG. 7 show embodiments with two readout stages 70.1, 70.2per pixel 50. Here, two sense diffusions 62.1, 62.2 (or 62.3, 62.4,respectively) are short-circuited to one sense node 63.1 (or 63.2),which can be accessed from two sides. This enables the readout of twosampling points per readout stage without moving the sampled valuesthrough the light-sensitive CCD gates. It is true that the sensediffusion 62.1–62.4 will get a larger capacitance (less than a factortwo) and, hence, a worse conversion factor (voltage increase perelectron); however, this drawback can be tolerated. It is important tomention that fixed-pattern noise due to the use of two readout stages70.1, 70.2 per pixel 50 is not a problem. This is because of thesubtraction of the balanced-mode sampling points in the evaluationalgorithm of Eq. (3). Fixed-pattern noise mainly adds an offsetcomponent to the pixel values. This offset disappears after thesubtraction. We call this “pair-wise offset-free charge integration”.

In the following, various methods for operating a TOFdistance-measurement system are discussed.

FIGS. 8 and 9 illustrate the operation principle of a TOF distancemeasurement system according to the state of the art. A modulated lightsource (or transmitter) 101 is normally located near a detector (orreceiver) 103. Since the optical power density on an illuminated objector target 102 decreases with the square of the distance L between theillumination source 101 and the target 102, the received intensityI_(rec) on the detector 103 also decreases with the square of the targetdistance L. With the transmission of a light pulse 104, a switch 105 inthe receiver 103 opens. The switch 105 closes with the end of the lightpulse 104. The amount of light integrated in the receiver 103 depends onthe overlap of the time window 107 defined by the ON time of the switch105 and the delayed time window 106 of ON time of the received lightpulse 104. Both ON time of the switch 105 and pulse width are chosen tohave the same length T. Thus, targets 102 with zero distance (L=0)receive the full amount of the light pulse 104, the complete light pulse104 is integrated. Targets 102 farther away from the light source 101only integrate a fraction of the light pulse 104. Only distancesL<L_(max) within the propagation range of the light pulse 104, definedby half the product of the pulse width T and the light velocity c, canbe measured. The amount of received light I_(rec) decreases with thesquare of the target distance L to the emitting active illuminationsource 101, as illustrated in FIG. 9. The prior-art shutter operationleads to an additional distance-dependent attenuation of the integratedreceived signal.

A first mode of operation according to one aspect of the invention isillustrated in FIGS. 10 and 11. According to this method, which can becalled the “inverse shutter operation”, the switch 105 is started at theend of the light-pulse transmission or even later. Targets 102 fartheraway from the range camera integrate a longer period of the light pulse104 than targets 102 near the range camera. Together with theattenuation of the back-scattered light from a diffusely reflectingtarget 102, which is proportional to the square of the distance L(assuming a Lambert-reflecting target), the resulting integrated signalshare of the light pulse now only decreases linearly with the distance Lof the target 102 (cf. FIG. 11).

FIGS. 12 and 13 show a second mode of operation according to one aspectof the invention. This method uses a falling-ramp pulse 104 instead of asquare pulse and combines this operation with the “inverse shutteroperation” explained with respect to FIGS. 10 and 11. This results in anintegrated signal that does not depend on the target's distance L, onlyon its remission coefficient, if the target 102 is a Lambert reflector.

1. A device for spatially resolved photodetection and demodulation oftemporally modulated electromagnetic waves, comprising: aone-dimensional or two-dimensional array of pixels (50), each pixel (50)comprising: at least two transducer elements (51.1, 51.2) fortransducing incident electromagnetic radiation into an electric signal,each transducer element (51.1, 51.2) being associated with: at least onestorage element (54.1–54.4) for storing said electric signal, said atleast one storage element (54.1–54.4) being inaccessible or insensitiveto incident electromagnetic radiation; wherein each pixel (50) comprisesan optical element (41) for spatially averaging the electromagneticradiation incident on the pixel (50) and equally distributing theaveraged electromagnetic radiation onto said transducer elements (51.1,51.2) of the pixel (50).
 2. The device according to claim 1, whereinsaid optical element (41) comprises at least one of a refractive elementand a diffractive optical element.
 3. The device according to claim 1,wherein each pixel (50) has two transducer elements (51.1, 51.2).
 4. Thedevice according to claim 1, wherein each transducer element (51.1) isassociated with two storage elements (54.1, 54.2).
 5. The deviceaccording to claim 1, wherein said arrangement of pixels (50) isrealized in CCD or in CMOS-APS/CCD technology.
 6. A method for spatiallyresolved photodetection and demodulation of temporally modulatedelectromagnetic waves, comprising the steps of: impingingelectromagnetic radiation onto a one-dimensional or two-dimensionalarray of pixels (50); spatially averaging the electromagnetic radiationincident on each pixel (50); equally distributing the averagedelectromagnetic radiation onto at least two transducer elements (51.151.2) of the pixel (50); transducing the equally distributed averagedelectromagnetic radiation incident on each pixel (50) into at least twoelectric signals in at least two transducer elements (51.1, 51.2); and,storing each of said at least two electric signals in at least onestorage element (54.1–54.4).
 7. The method according to claim 6, whereinsaid at least two transducer elements (51.1, 51.2) of a pixel (50) areoperated with a 90° phase shift with respect to each other.
 8. Themethod according to claim 6, wherein each pixel (50) is addressed andread out individually.